ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information

Object detection in wide area motion imagery (WAMI) has drawn the attention of the computer vision research community for a number of years. WAMI proposes a number of unique challenges including extremely small object sizes, both sparse and densely-packed objects, and extremely large search spaces (large video frames). Nearly all state-of-the-art methods in WAMI object detection report that appearance-based classifiers fail in this challenging data and instead rely almost entirely on motion information in the form of background subtraction or frame-differencing. In this work, we experimentally verify the failure of appearance-based classifiers in WAMI, such as Faster R-CNN and a heatmap-based fully convolutional neural network (CNN), and propose a novel two-stage spatio-temporal CNN which effectively and efficiently combines both appearance and motion information to significantly surpass the state-of-the-art in WAMI object detection. To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the convolutional architecture, and proposes regions of objects of interest (ROOBI). These ROOBI can contain from one to clusters of several hundred objects due to the large video frame size and varying object density in WAMI. The second stage (FoveaNet) then estimates the centroid location of all objects in that given ROOBI simultaneously via heatmap estimation. The proposed method exceeds state-of-the-art results on the WPAFB 2009 dataset by 5-16% for moving objects and nearly 50% for stopped objects, as well as being the first proposed method in wide area motion imagery to detect completely stationary objects.Comment: Main paper is 8 pages. Supplemental section contains a walk-through of our method (using a qualitative example) and qualitative results for WPAFB 2009 datase

Algorithms and Applications of Novel Capsule Networks

Convolutional neural networks, despite their profound impact in countless domains, suffer from significant shortcomings. Linearly-combined scalar feature representations and max pooling operations lead to spatial ambiguities and a lack of robustness to pose variations. Capsule networks can potentially alleviate these issues by storing and routing the pose information of extracted features through their architectures, seeking agreement between the lower-level predictions of higher-level poses at each layer. In this dissertation, we make several key contributions to advance the algorithms of capsule networks in segmentation and classification applications. We create the first ever capsule-based segmentation network in the literature, SegCaps, by introducing a novel locally-constrained dynamic routing algorithm, transformation matrix sharing, the concept of a deconvolutional capsule, extension of the reconstruction regularization to segmentation, and a new encoder-decoder capsule architecture. Following this, we design a capsule-based diagnosis network, D-Caps, which builds off SegCaps and introduces a novel capsule-average pooling technique to handle to larger medical imaging data. Finally, we design an explainable capsule network, X-Caps, which encodes high-level visual object attributes within its capsules by utilizing a multi-task framework and a novel routing sigmoid function which independently routes information from child capsules to parents. Predictions come with human-level explanations, via object attributes, and a confidence score, by training our network directly on the distribution of expert labels, modeling inter-observer agreement and punishing over/under confidence during training. This body of work constitutes significant algorithmic advances to the application of capsule networks, especially in real-world biomedical imaging data

Deformable Capsules for Object Detection

In this study, we introduce a new family of capsule networks, deformable capsules (DeformCaps), to address a very important problem in computer vision: object detection. We propose two new algorithms associated with our DeformCaps: a novel capsule structure (SplitCaps), and a novel dynamic routing algorithm (SE-Routing), which balance computational efficiency with the need for modeling a large number of objects and classes, which have never been achieved with capsule networks before. We demonstrate that the proposed methods allow capsules to efficiently scale-up to large-scale computer vision tasks for the first time, and create the first-ever capsule network for object detection in the literature. Our proposed architecture is a one-stage detection framework and obtains results on MS COCO which are on-par with state-of-the-art one-stage CNN-based methods, while producing fewer false positive detections, generalizing to unusual poses/viewpoints of objects

Clusternet: Detecting Small Objects In Large Scenes By Exploiting Spatio-Temporal Information

Object detection in wide area motion imagery (WAMI) has drawn the attention of the computer vision research community for a number of years. WAMI proposes a number of unique challenges including extremely small object sizes, both sparse and densely-packed objects, and extremely large search spaces (large video frames). Nearly all state-of-the-art methods in WAMI object detection report that appearance-based classifiers fail in this challenging data and instead rely almost entirely on motion information in the form of background subtraction or frame-differencing. In this work, we experimentally verify the failure of appearance-based classifiers in WAMI, such as Faster R-CNN and a heatmap-based fully convolutional neural network (CNN), and propose a novel two-stage spatio-temporal CNN which effectively and efficiently combines both appearance and motion information to significantly surpass the state-of-the-art in WAMI object detection. To reduce the large search space, the first stage (ClusterNet) takes in a set of extremely large video frames, combines the motion and appearance information within the convolutional architecture, and proposes regions of objects of interest (ROOBI). These ROOBI can contain from one to clusters of several hundred objects due to the large video frame size and varying object density in WAMI. The second stage (FoveaNet) then estimates the centroid location of all objects in that given ROOBI simultaneously via heatmap estimation. The proposed method exceeds state-of-the-art results on the WPAFB 2009 dataset by 5-16% for moving objects and nearly 50% for stopped objects, as well as being the first proposed method in wide area motion imagery to detect completely stationary objects

Free flap monitoring using an implantable Doppler probe

Although clinical observation is the gold standard, the ideal free flap monitoring device has not been identified. The purpose of the present study was to review the first 14 months of experience using an implantable 20-MHz ultrasonic Doppler probe to monitor the microvascular anastamoses of free tissue transfers. Twenty-five flaps in 23 patients, with an average age of 51 years (age range 18 to 81 years), were performed. Probes were secured downstream of the venous anastamosis using a silicone-polyfluorotetraethylene sleeve. Doppler sounds were transduced before the flap was inset. Monitoring by nursing staff included conventional techniques (temperature, colour, capillary refill) and continuous Doppler flow monitoring. Dynamic diagnostic testing for anastomotic patency was facilitated by applying manual pressure on the flap to increase venous flow (the audible 'whoosh' sign) and valsalva manoeuvre to impede venous return momentarily (the 'heave' sign). Intraoperative vessel kinking, hematoma formation occluding venous outflow, and venous thrombosis were detected in four cases before concluding the procedure and corrected. Rapid, immediate cessation of audible flow was detected postoperatively in three of 25 flaps. Re-exploration (re-exploration rate 12%) led to salvage of all three flaps (salvage rate 100%). It was concluded from this study that flap re-exploration was prevented in four cases (16%) because of intraoperative use of the implantable Doppler probe. Earlier detection of flap compromise perioperatively is thought to have contributed to the 100% salvage rate and to the 100% flap survival rate in the first 25 cases in which the implantable Doppler probe was used. Key Words: Implantable Doppler probe; Free flap; Monitor Surveillance des greffes libres à l'aide d'une sonde Doppler implantable RÉSUMÉ : Même si l'observation clinique reste la norme en matière de surveillance, le dispositif idéal de surveillance des greffes libres n'a pas encore été trouvé. La présente étude a pour but de faire état des 14 premiers mois d'expérience de surveillance des anastomoses microvasculaires de greffes libres à l'aide d'une sonde Doppler implantable à ultrasons de 20 MHz. Vingt-cinq greffes ont été pratiquées sur 23 patients dont l'âge variait de 18 à 81 ans (âge moyen : 51 ans). Les sondes ont été fixées sur les anastomoses veineuses du côté des patients à l'aide d'un manchon de silicone et de polyfluorotétraéthylène. On a procédé à la transduction des signaux Doppler avant la mise en place du greffon. La surveillance par le personnel infirmier faisait appel aux techniques usuelles d'observation (température, couleur, remplissage capillaire) et à l'examen du signal Doppler continu. L'application d'une pression manuelle sur le greffon pour accroître le débit veineux (signe sonore « whoosh ») et la manoeuvre de Valsalva pour gêner temporairement le retour veineux (signe du « dôme ») facilitaient l'examen diagnostique dynamique de la perméabilité des anastomoses. Dans quatre cas, on a décelé et corrigé, avant la fin de l'intervention, la présence d'une plicature vasculaire, la formation d'un hématome bloquant la circulation veineuse et la présence d'une thrombose veineuse. La perte rapide et immédiate du signal sonore en phase postopératoire a été perçue dans trois cas. Une nouvelle exploration (taux de reprise : 12 %) a permis de sauver les trois greffons (taux de sauvetage : 100 %). Aussi les résultats nous permettent-ils de conclure que le recours à la sonde Doppler implantable en cours d'intervention a permis d'éviter une nouvelle exploration du greffon dans quatre cas (16 %) et que la détection précoce d'atteinte à l'intégrité du greffon en phase périopératoire par la sonde a joué un rôle dans le taux de sauvetage de 100 % et le taux de survie du greffon de 100 % dans les 25 premiers cas